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Abstract 



■ We are motivated by the problem of designing a simple distributed algorithm for Peer-to-Peer 

streaming applications that can achieve high throughput and low delay, while allowing the neighbor 
set maintained by each peer to be small. While previous works have mostly used tree structures, 
our algorithm constructs multiple random directed Hamiltonian cycles and disseminates content 
over the superposed graph of the cycles. We show that it is possible to achieve the maximum 
streaming capacity even when each peer only transmits to and receives from 0(1) neighbors. 
Further, we show that the proposed algorithm achieves the streaming delay of O(logiV) when 
the streaming rate is less than (1 — 1/K) of the maximum capacity for any fixed constant K > 2, 
where N denotes the number of peers in the network. The key theoretical contribution is to 
characterize the distance between peers in a graph formed by the superposition of directed random 
Hamiltonian cycles, in which edges from one of the cycles may be dropped at random. We use 
Doob martingales and graph expansion ideas to characterize this distance as a function of N, with 
£SJ ' high probability. 
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Peer-to-Peer Networks, Streaming Media, Delay Analysis, Random Graph Theory 

t^-. ■ I. Introduction 

Dissemination of multimedia content over the Internet is often accomplished using a central 
server or a collection of servers which disseminate the data to all clients interested in the content. 
Youtube is an example of such a model, where multiple large-capacity servers are used to meet 
the download demands of millions of users. In contrast, in a peer-to-peer (P2P) network, a small 
(low-capacity) server uploads the content to a small number of clients, and these clients and 
all other clients in the network then exchange content among themselves. The P2P approach is 
scalable since the network utilizes the upload capacities of all the clients (commonly known as 
peers) in the network: as more peers join the network, the download requirement increases but 
the available capacity also increases proportionally. 

In this paper, we are interested in designing P2P networks where each peer needs to keep track 
of only a small number of other peers in the network. Restricting the neighborhood size of each 
peer reduces the administrative overhead for the peers. Therefore, one of the key challenges is 
to design an algorithm to decide which peers should belong to the same neighborhood. Such 
algorithms are called "pairing" algorithms since they pair peers to be neighbors. The pairing 
algorithm must be lightweight, i.e., when new peers enter the network or when existing peers exit 
the network, the algorithm should incur low overhead to readjust the pairing relationships in the 
neighborhood. In this paper, we use a pairing algorithm proposed in (TJ. However, in |[T) the fact 
that the pairing algorithm produces a topology with small diameter is sufficient for their purpose. 
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Here, this fact is not sufficient to ensure that the P2P streaming network has high capacity and 
low delay. 

The pairing algorithm mentioned in the previous paragraph determines the topology of the net- 
work. Given the topology, the network must then decide how to disseminate content in the network 
to achieve the maximum possible capacity and low delay. Multimedia content is often divided 
into chunks and thus, the content dissemination algorithm is also called the chunk dissemination 
algorithm in the literature. Chunk dissemination is accomplished by a peer in two steps in each 
timeslot: the peer has to select a neighbor to receive a chunk (called neighbor selection) and then 
it has to decide which chunk it will transmit to the selected neighbor (called chunk selection). 
Thus, the practical contributions of the paper can be summarized as follows: we present a low- 
complexity, high-throughput and low-delay algorithm for pairing, neighbor selection and chunk 
selection in real-time P2P streaming networks. We emphasize that the goal of this paper is to study 
real-time data dissemination in P2P networks. This is in contrast to stored multimedia content 
dissemination (which is the bulk of Youtube's data, for example) or file-transfer applications (such 
as in BitTorrent). 

Our approach for pairing results in a graph formed by the superposition of multiple random 
directed Hamiltonian cycles over a given collection of nodes (peers). We will see that the perfor- 
mance analysis of our algorithms requires us to understand the distance (the minimum number of 
hops) from a given peer to all other peers in the graph. The main theoretical contribution of the 
paper is to characterize these distances with high probability through a concentration result using 
Doob martingales. Using this result, we show that our algorithm achieves 0(logiV) delay with 
high probability, when the streaming rate is less than or equal to (1 — j^) of the optimal capacity 
for any constant K > 2, where N denotes the number of peers in the network. 

This paper is organized as follows. In Section [II] we review prior work in the area of real- 
time P2P networks. In Section [Hi] we provide two examples to help the reader understand the 
advantage of using random Hamiltonian cycles. In Section IIVI we present our P2P algorithm 
that constructs random Hamiltonian cycles and disseminates content over the cycles in a fully 
distributed manner. In Section [V] we consider the streaming rate that can be achieved under our 
algorithm. In Sections |VI] and IVIII we analyze the delay to disseminate chunks to all peers under 
our algorithm. In Section [Villi we conclude the paper. 

II. Related Work 

We briefly review prior work in the area of real-time P2P networks. Prior work in the area 
can be broadly categorized as designing one of two types of networks: a structured P2P network 
or an unstructured P2P network. The structured P2P streaming approach focuses on constructing 
multiple overlay spanning trees that are rooted at the source |2]-||9]]- In this approach, the real- 
time content arriving at the source is divided into multiple sub-streams and each sub-stream is 
delivered over one of the trees. Since this approach uses the tree structure, connectivity from the 
source to all peers is guaranteed. By managing the tree depth to be O(logiV), this approach can 
guarantee O(logiV) delay to disseminate a chunk of each sub-stream to all peers. However, the 
fundamental limitation of the structured P2P streaming is vulnerability to peer churn. It is well 
known that the complexity of constructing and maintaining 0(log 7V)-depth trees grows as N 
increases. Therefore, in a highly dynamic P2P network where peers frequently join and leave the 
network, the structure approach is not scalable. 

Unstructured P2P networks overcome this vulnerability to peer churn. In unstructured P2P 
networks, peers find their neighboring peers randomly and get paired with them locally. As a 
neighboring peer leaves, a peer chooses another peer randomly as its new neighboring peer. 
Due to the distributed fashion of this peer pairing, unstructured P2P networks are robust to peer 
churn, unlike the structured P2P networks. However, the fundamental limitation of unstructured 
P2P networks is weak connectivity. Since peers are paired randomly without considering the entire 
network topology, there may be some peers that are not strongly connected from the source, which 
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results in poor throughput and delay. To ensure full connectivity in this approach, it is required 
that every peer should be paired with O(logTV) neighboring peers iflOl . or should constantly 
change their neighbors to find neighbors providing a better streaming rate ifTTl . However, in these 
approaches, delay performance is hard to guarantee because chunks have to be disseminated over 
an "unknown" network topology. 

Another interesting line of work has studied gossip-based algorithms that disseminate informa- 
tion to all peers in a fashion similar to the spread of epidemics. By studying the dissemination 
delay under these gossip-based algorithms, we can analyze the delay for peers to disseminate 
chunks to all peers in a P2P network. The seminal work in lfl2ll shows that gossiping requires 
0(logiV) time with high probability to disseminate a single chunk from the source to all peers. 
When there is a sequence of chunks arriving at the source, the latest-blind algorithm proposed 
in lfl3l is proven to deliver (1 — e _1 ) fraction of chunks to all peers with 0(log TV) delay with 
high probability. Later work in lfT4l proposed the latest useful algorithm that can deliver almost 
all chunks with 9 (log TV) delay with high probability. However, the basic assumption in this line 
of work is that the network is a complete graph where every peer has TV — 1 outgoing edges to all 
other peers. For an incomplete graph, it is shown in ifTBl that gossip-based algorithms can achieve 
0(log TV) delay, when the matrix representing the connectivity between peers is doubly stochastic 
and symmetric. However, only a small fraction of the optimal throughput can be guaranteed with 
6 (log TV) delay. 

We address all the aforementioned limitations using multiple random Hamiltonian cycles. While 
the structure of Hamiltonian cycles provides us with full connectivity from the source to all 
peers, random pairing within each cycle enables peers to cope with peer churn. Furthermore, the 
proposed chunk dissemination algorithm guarantees (log TV) delay required for each chunk to 
be disseminated to all the peers for a near optimal throughput. One may be concerned about using 
cycles because the diameter TV of a cycle could result in poor delay performance. However, we 
address this concern in the next section. 



In a delay-sensitive application, such as P2P streaming, cycles (or line topologies) have been 
considered to be undesirable since their diameter is TV — 1, where the diameter of a directed graph 
is defined as the maximum distance between any pair of nodes. Delivering information from a 
node to all the other nodes over a cycle requires TV — 1 successive transmissions, which results in 
0(TV) delay. In this section, we consider two examples which show that one can use a superposed 
graph of multiple cycles as an alternative to the tree structure for information dissemination. 



Fig. 1. Two random Hamiltonian cycles (TV = 12) generated by independent permutations of nodes 1, 2, ■ ■ ■ , TV. 



III. Independent Random Hamiltonian Cycles 




Consider TV nodes numbered 1, 2, ■ ■ • , TV. By permuting these nodes, we can make a random 
Hamiltonian directed cycle as shown in Fig. |l(a)| (Since all the graphs that we will consider 
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are directed graphs, we will skip mentioning "directed" from now on.) Make another random 
Hamiltonian cycle by independently permuting the N nodes as shown in Fig. |l(b)| Clearly, the 
diameter of each cycle is N — 1. An interesting question is the following: "if we superpose both 
cycles, what is the diameter of the superposed graph?" Interestingly, the diameter significantly 
reduces from Q(N) to 0(log N) with high probability in the superposed graphQ 

Next, we consider a further modification of the two random cycle model. From the second 
cycle (Fig. |l(b)| i, we remove each edge with some probability < q < 1 independently. If we 
superpose the first cycle and the remaining edges in the second cycle, what will be the diameter 
of the graph? Since we have removed around qN edges from the second cycle, the diameter will 
certainly increase. However, we will show in a later section that the order of the diameter still 
remains ®(\ogN). 

In the rest of this paper, we show how these properties of random superposed cycles can be 
used to construct a P2P network that can achieve high throughput and low delay. 

IV. System Model 

We assume that time is slotted, and every peer (including the source peer) in the network 
contributes a unit upload bandwidth, i.e., each peer can upload one chunk per timeslot. In this 
case, it is well known that the maximum streaming rate (the maximum reception rate guaranteed 
to each peer) is approximately one for a large network because the total upload bandwidth N 
contributed by all peers (including the source) has to be shared by N — 1 peers (excluding the 
source) H-Q. Due to the limited communication and computation overheads, we assume that 
each peer can only communicate with a constant number of neighbors, which does not increase 
with the network size. We assume that there is peer churn, so that the topology is dynamic as 
new peers join or existing peers leave. 

We now present our P2P streaming algorithm which consists of a peer-pairing algorithm and a 
chunk-dissemination algorithm. For convenience, we use the term chunk dissemination algorithm 
to describe the joint neighbor selection and chunk selection algorithms mentioned in the previous 
section. As mentioned earlier, our pairing algorithm is the same as in [T]. But it is important to 
understand the pairing algorithm to proceed further. Therefore, we present it next. 

A. Peer Pairing Algorithm 

Under our peer-pairing algorithm, every peer has M > 2 incoming edges and M outgoing 
edges as shown in Fig. [2] We number the incoming edges of each peer as the first, second, 
Af-th incoming edges of the peer and number its outgoing edges as the first, second, M-th 
outgoing edges. The peer where the m-th outgoing edge ends is called the m-th child, and the 
peer where the m-th incoming edge begins is called the m-th parent. We assume that the M 
outgoing edges of a peer may end at the same peer, so that the number of the children of a peer 
could be less than M. Similarly, the M incoming edges of a peer may begin at the same peer, 
so that the number of the parents of a peer could be less than M. Under our algorithm, every 
peer receives chunks from its parents over its incoming edges and transmits received chunks to 
its children over its outgoing edges. 

We next describe how every peer establishes its M incoming and M outgoing edges. 
Initially, the network consists of only two peers, the source peer (which we call peer 1) and 
the first peer to arrive at the network (which we call peer 2), as shown in Fig. |3(a)| Each peer 
establishes its first, second, Af-th outgoing edges to the other peer, so that these edges are the 
first, second, Af-th incoming edges of the other peer. Letting V be the set of current peers, we 

'in random graph theory, it is shown that the superposition of two undirected random Hamiltonian cycles has a 
distribution similar to an undirected random regular graph 1161 . This regular graph is known to have ©(log N) diameter 
with high probability 1171 . Combining both results, we can infer that the superposition of two random undirected 
Hamiltonian cycles has 0(log./V) depth with high probability. From this, it is not very difficult to obtain a similar 
result for directed cycles. However, in 1181 . we establish this result more directly. 
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Parents 




Fig. 2. Basic structure of the peer-pairing algorithm for M = 2: Every peer maintains M incoming edges and M 
outgoing edges. The peers where the outgoing edges end are called children, and the peers where the incoming edges 
begin are called parents. Every peer v receives chunks from parents and transmits chunks to children. 

define E m to be the set of all m-th edges, i.e., E, n = £ V 2 \ j is the rn-th child of i}, for 

m = 1, 2, ■ • • , M. Initially, E m is given by {(1, 2), (2, 1)} for all m because there are only two 
peers. We define L m = (V, E m ) to be the digraph consists of the peer set V and the ?n-th edges, 
and call it layer m for m = 1, 2, • • • , M. Layer m represents the pairing between every peer and 
its m-th child. By superposing the M layers, the current network topology can be expressed as a 
multi-digraph L* = (V, E*), where E* is a multiset defined as E* = {E x U E 2 L), • • • , E M }- 




(a) Initial graph: layer 1 (left), layer 2 (center), and the superposed graph (right) 




new new new 

(b) Peer 4 arrives: layer 1 (left), layer 2 (center), and the superposed graph (right) 




(c) Peer 2 leaves: layer 1 (left), layer 2 (center), and the superposed graph (right) 



Fig. 3. Peer pairings for M — 2; (a) Initially, peer 1 takes peer 2 as its first and the second children, and vice versa. 
(b) When peer 4 arrives to the network with existing peers 1, 2, and 3, it independently chooses an edge (marked 'x') 
from each layer uniformly at random and breaks into the chosen edges, (c) When peer 2 leaves, its incoming edge and 
outgoing edge (marked 'x') in each layer are reconnected. 

When a new peer v arrives, this peer independently chooses an edge from each layer uniformly 
at random and breaks into the chosen edges as Fig. |3(b)| Specifically, if the peer v arrives and 
randomly chooses (p m ,c m ) £ E m from layer m, the peer v becomes a new m-th child of peer 
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p m and becomes a new m-th parent of peer c rn . Each layer m will then be updated as 

V ^VU{v} 

E m <=E m U {(p m , v), (v, c m )} \ (p m ,Cm), Vm. 

In practice, this edge-breaking can be easily implemented. If a new peer arrives, it contacts a 
server to register its IP address. The server then chooses M(l + a) IP addresses uniformly at 
random with repetition and returns them to the peer. Here, a > is used in practice in case some 
peers are not reachable for some reason. But for the purpose of analysis later, we assume that 
a = and all peers are reachable. Among these addresses, the peer contacts M reachable peers 
and breaks into their first, second, M-th outgoing edge, respectively. 

When an existing peer v leaves, its parents and children will lose one of their neighbors as 
shown in Fig. |3(c)| Let p m and c m be the parent and the child, respectively, of peer v in layer 
to, i.e., (p m ,v), (v,c m ) £ E m . (It is easy to see that every peer always has exactly one parent 
and one child in each layer.) The parent p m in each layer m then directly contacts the child c m 
in the same layer and takes the child as its new m-th child as shown in Fig. |3(c)| In this case, 
the topology will change as follows: 

V ^V\{v} 

E m <=E m U (p m ,c m ) \ {(p m ,v), (v,c m )}, Vto, 

In practice, this edge-repairing can also be easily implemented. We let every peer remember 
the IP addresses of the closest decedents (or ancestors) within h > 2 hops in each layer. If its 
closest k < h descendants (ancestors) in layer to leave the network simultaneously without any 
notification, the peer can contact the (k + l)-th closest descendent (ancestors) and repair its m-th 
pairing without help from the server. In this case, every peer needs to remember M ■ h IP addresses 
of its descendants (ancestors), which can be locally updated through neighbors. 

At any given time, the network topology L* that has been constructed by the pairing algorithm 
satisfies the property stated in the following lemma. The lemma and its proof are straightforward, 
given the pairing algorithm, but we present them below to highlight their importance to the analysis 
in the rest of the paper. 

Lemma 1: L m representing each layer m is a directed hamiltonian cycle, i.e., every peer has 
exactly one incoming and one outgoing edge in each layer, and all the edges in E m form a single 
directed cycle. Hence, the superposed graph L* is an A/-regular multi-digraph, i.e., every peer 
has exactly M incoming edges and M outgoing edges. 

Proof: Initially, two peers form a single cycle in each layer. When we add a new peer to 
each layer consisting of a single cycle, the new peer simply breaks into an existing edge in a 
layer, maintaining the existing cycle. When we remove a peer from each layer, its incoming edge 
and its outgoing edge are reconnected, which also maintains the cycle. Hence, when peers join 
or leave, the cycles built initially do not vanish, but they expand or shrink in size. Hence, each 
layer is always a cycle graph. ■ 

Lemma [T] implies that, under the pairing algorithm, each peer needs to communicate only with 
M parents and M children. Hence, the communication/computation overhead to maintain multiple 
TCP or UDP sessions does not increase with the network size. 

Remark: The pairing algorithm is fully distributed, except for the information provided by a 
central server to identify a few other peers in the network. The server only maintains the list 
of registered peers and their IP addresses, which need not be updated frequently. If some peers 
in the list do not exist in the network any more, the server may send the IP address of such a 
peer to a new peer. However, this does not affect the pairings of the new peer because the new 
peer will only contact M reachable addresses among the M(l + a) received addresses. A central 
server to perform such minimal functionalities is usually called a tracker, and is used by most 
P2P networks. In our analysis, we do not consider the details of the information sharing between 



7 



the tracker and the peers. We simply assume that M random addresses are provided to a new 
peer to enable it to execute the pairing algorithm. 

B. Chunk Dissemination Algorithm 

While the pairing algorithm determines the network topology, the chunk dissemination algorithm 
determines how chunks are disseminated over a given topology. We here present our chunk 
dissemination algorithm that can provide provable throughput and delay bounds. 

Assume that the source generates at most one chunk during every timeslot, except timeslots 
0, K, 2K, ■ ■ ■ for some integer K > 2. Since at most K — 1 chunks are generated during every 
K timeslots, the maximum chunk-generating rate under our algorithm is (1 — 1/K). We call the 
chunk generated at timeslot t chunk t. Suppose there are K — 1 predetermined colors, numbered 
color 1, color 2, color K — 1 and we color each chunk t with color (t mod K). In other 
words, the chunks are colored from 1 through K — 1, and then again starting from 1, with the 
process repeating forever. We call the chunk with color k simply a color- k chunk. If a color- k 
chunk is generated at time t at the source, then color k chunks are also generated at time t + K, 
t + 2K and so on. If chunks are not generated periodically in this manner, then a smoothing 
buffer has to be used at the source to ensure that only K — 1 chunks are periodically generated 
for every K timeslots, and any other additional chunks are stored for later transmission. Thus, 
there will be a queueing delay at the source for storing the additional chunks which we ignore 
since our goal here is to characterize the scaling behavior of the end-to-end transmission delay 
from the source to all peers as a function of N. 

Recall that every peer can upload at most one chunk to one other peer in a timeslot. At the 
beginning of each timeslot, every peer schedules one of its outgoing edges, i.e., the peer selects 
an outgoing edge and uploads a chunk over that edge. Specifically, every peer i shares the same 
scheduling vector A = (Ai,A2,--- ,Xk), where Afc € {1,2, ••• ,M — 1} for all k < K and 
Ajr = M. Peer i schedules its outgoing edges, cycling through the elements in the scheduling 
vector. For example, if M = 3 and A = (1, 2, 1, 3), every peer repeats scheduling its first, second, 
first, and third outgoing edges sequentially. We note that the scheduling round of a peer need not 
be synchronized with the other peers, i.e., at a give timeslot, peers may schedule different types 
of outgoing edges. 

Suppose peer v schedules the fc-th edge in the scheduling vector (i.e., the outgoing edge in layer 
Xk or equivalently the Afc-th outgoing edge) at the beginning of timeslot t. Let Q v .k(t) be the set 
of the color-A: chunks that peer v have received before timeslot t for k < K. If k < K, peer v 
chooses the chunk from Q Vt k{t) that was generated most recently (called the latest chunk) and 
uploads this chunk over the scheduled edge, regardless of whether or not the other end possesses 
the chunk. If k = K, the peer transmits the latest chunk in Q v ^( v )(f), where p(v) is a random 
variable uniformly chosen from {1, 2, • • • , K — 1} when the peer joined the network. We call p(v) 
the coloring decision of peer v. During K timeslots of a scheduling round, peer v will transmit 
the latest chunks with color 1, color 2, color K — 1, and color p(v) over the Ai-st, A2-nd, • • • , 
Aa'-A outgoing edges, respectively. Since the scheduling rounds of peers are asynchronous, peers 
may transmit chunks with different colors at a given timeslot. Note that when a peer receives a 
chunk, this chunk will be unavailable for uploading till the next timeslot. Furthermore, we have 
assumed implicitly that only the latest chunk with each color is available for uploading at a peer. 
Thus, if a color-fc chunk that is generated later than the latest chunk in Q Vt k(t) of peer v arrives 
at peer v at timeslot i, the peer will not upload all the chunks received before timeslot t. We 
will show later that all chunks are delivered to all the peers despite the fact that older chunks 
are discarded. In other words, we will prove that the older chunks have already disseminated by 
a peer by the time they are discarded and so are no longer necessary from the point of view of 
data dissemination (although they may be retained for playout at the peer). 

We have presented our chunk-dissemination algorithm running on top of the pairing algorithm. 
Besides our algorithm, other chunk dissemination algorithms, such as the random useful algorithm 
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|fl9l , the latest-blind algorithm |fl3l , and the latest-useful algorithm fl4l . can be potentially used 
over the network topology that is constructed by the pairing algorithm. Our performance analysis 
is, however, only for the chunk dissemination algorithm proposed here. 

C. Streaming Rate and Delay 

Our P2P algorithm will be evaluated using two metrics: streaming rate and delay. 

Streaming Rate: What is the streaming rate achieved by our P2P algorithm? The streaming 
rate is defined as the chunk reception rate guaranteed to all peers. When peers contribute unit 
bandwidth, the total upload bandwidth N contributed by all peers (including the source) has to 
be shared by N — 1 peers (excluding the source). Thus, the download bandwidth per peer cannot 
exceed -Jzj, which is approximated to one for large TV. Hence, the optimal streaming rate is 
close to one for a large network. In Section [V] we will show that our algorithm disseminates all 
the chunks to all peers, and achieves a streaming rate of 1 — which is arbitrarily close to the 
optimal streaming rate for sufficiently large K. 

Dissemination Delay: What is the delay that can be achieved by our P2P algorithm? When each 
peer is allowed to disseminate chunks only to a constant number of neighbors, as in a real P2P 
topology, the fundamental limit of the delay required to disseminate a chunk to all peers is known 
to be 6 (log iV) II This limit is a lower bound on the delay to disseminate multiple chunks because 
the contention between multiple chunks at a peer can only increase the dissemination delay. In 
Section [VTl we show that our algorithm achieves this fundamental limit, i.e., every chunk arriving 
at the source at rate (1 — -4) is disseminated to all peers within 0(logiV) timeslots with high 
probability under our algorithm. 

V. Throughput and Delay Analysis 

In this section, we show that our algorithm achieves the streaming rate of (1 — -^), i.e., each 
chunk arriving at the source at rate (1 — -4) can be disseminated to all peers by our algorithm. 
To this end, we first characterize the graph over which color-fc chunks are disseminated. We then 
show that no color-fc chunks are dropped before being disseminated to all peers. 

As described in Section IIV-BI during every scheduling round of a peer v, peer v transmits 
the latest color-1 chunk, color-2 chunk, color-(/\ — 1) chunk, and color-/i(v) chunk over its 
Ai-st, A2-nd,..., A(A'-i)-st, M-th outgoing edges, respectively. Thus, color-fc chunks are delivered 
over the Afc-th outgoing edges from all peers (i.e., the edges in layer Afc) and the M-th outgoing 
edges from peers v with /i(t') = fc. If we define flow graph Gk (k = 1, 2, • ■ • , K — 1) to be 
the graph consisting of the edges carrying color-fc chunks, the flow graph can be expressed as a 
multi-digraph Gk = (V, E\ k U £M,k), where 

SM,k = {(i,j)eE M Hi)=k}. (1) 

Thus, color-fc chunks are disseminated over flow graph fc. (See the example of the flow graphs 
for K = 3 and M = 2 in Fig. [4(b)| . 

We next study how color-fc chunks are disseminated over flow graph Gk- Recall that if a color-fc 
chunk is generated at timeslot t, chunks t + K, t + 2K, • • • are all of color fc. We call these chunks 
later chunks of chunk t. Since our chunk dissemination algorithm transmits only the latest chunk 
of each color, if a peer has received both chunk t and a later chunk, the peer will not transmit 
chunk t any longer. Thus, if all the peers that have received chunk t have also received a later 
chunk, chunk t cannot be disseminated to the remaining peers. However, the following proposition 
shows that this scenario does not occur. 

Proposition 1: Under our algorithm, if a peer receives chunk t during timeslot I > t, this peer 
has received chunk t — K before or during timeslot I — K. 

2 If we trace the paths that a chunk has been transmitted, the paths form an arborescence with a bounded degree rooted 
at the source. Since this arborescence has at least 0(logA r ) depth, distributing a chunk to all peers requires at least 
0(logA r ) transmissions. 
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(a) Layer 1 (left) and layer 2 (right): The numbers on peers v in layer 2 are the coloring decisions 
fj,(v). 




(b) Flow graphs Gi (left) and G 2 (right) 



Fig. 4. A network with 6 peers named s (source), 2,3,...,6, where M = 2, K = 3, A = (1,1,2) and 
(fi(s), /i(2), • ■ ■ , fJ.(6)) = (1, 2, 2, 1, 1, 2). Flow graph Gi consists of the edges in the first layer and the edges (i, j) 
with = 1 in the second layer. Similarly, flow graph G-i consists of the edges in the first layer and the edges (i, j) 
with fi(i) = 2 in the second layer. 



Proof: Without loss of generality, fix t = 0. We prove by induction that if a peer receives 
chunk (with color fc) during timeslot Z > 0, it has received chunk — K before or during timeslot 
l-K. 

Initially (I = 0), chunk arrives at the source during timeslot 0. Since the coloring queue 
moves one color-fc chunk every K timeslots, chunk —K must have arrived at the dissemination 
queue of the source during timeslot —K. Since the source is the only peer that has chunk during 
timeslot 0, the statement is true for I = 0. 

We now assume that the statement is true for I <t' — l. We next show that the statement is 
also true for I = t'. Consider a particular peer j that receives chunk for the first time through an 
incoming edge during timeslot t' . This implies that peer i has received chunk for the first 
time during timeslot t" G {t' — K, t' — K+ 1, • • ■ , t' — 1}. (If t" > t', peer j cannot receive chunk 
during timeslot t'. If t" < t' — K, peer j must have received chunk during timeslot t' — K.) By 
the induction hypothesis, peer i must have received chunk —K before or during timeslot t" — K. 
Thus, chunk ~K has been the latest color-fc chunk to peer i from timeslot t" — K + 1 to timeslot 
t". Since edge is scheduled once for every K timeslots to deliver the latest color-fc chunk, 
the edge was scheduled during timeslot t' — K. Since t" — K + 1 < t' — K < t", chunk —K 
must have been transmitted during that timeslot over Thus, the statement is true for I = t'. 

By induction, if a peer first receives chunk during timeslot I, it has received chunk —K before 
or during timeslot I — K. ■ 
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Proposition Q] implies that if a peer receives chunk t, it has at least K timeslots (one scheduling 
round) to distribute the chunk to its children before a later chunk arrives. Since the peer schedules 
each outgoing edge in Gk exactly once during every K timeslots for transmitting color-fc chunks, 
the peer will transmit chunk t to its children in Gk before a later chunk arrives. Thus, every 
color-fc chunk arriving at the source can be disseminated to all the peers that are connected from 
the source in Gk, i.e., there exists a path from the source to the peers in Gk- Since every flow 
graph Gk contains layer Afe, which is a Hamiltonian cycle, every peer is connected from the 
source. Thus, all chunks arriving at rate (1 — 1/K) can eventually be disseminated to all peers 
under our chunk-dissemination algorithm. 

In streaming applications, this throughput analysis is meaningless without a delay guarantee. 
We next consider how fast each color-fc chunk is disseminated over flow graph Gk- 

Lemma 2: Let dk(v) be the shortest distance from the source to peer v in flow graph Gk- If 
a color-fc chunk arrives at the source during timeslot t, peer v receives the chunk by timeslot 
t + Kd k (v). 

Proof: If a peer receives a color-fc chunk, it transmits the chunk to its children in Gk during 
next K timeslots by Proposition Q] Thus, the time until peer v receives the chunk through the 
shortest path in Gk from the source to peer v does not exceed Kdk{v). ■ 
Lemma [2] shows that the delay required to disseminate a color-fc chunk to peer v is upper 
bounded by K ■ dk(v) timeslots. If we call the maximum distance max,j e y dk(v) the depth d^ 
of Gk, the delay to disseminate a color-fc chunk to all peers is upper bounded by Kd k timeslots. 
Thus, if d* k is 0(log N) for all k = 1, 2, • • ■ , K — 1, the dissemination delay of our algorithm is 
upper bounded by O(logiV) timeslots. In the next section, we prove that this is true with high 
probability. 

VI. Depth of Flow Graphs 

In this section, we consider the depth d* k of each flow graph Gk- Since all the layers are random 
graphs, which are affected by the history of past peer churn, the corresponding flow graph is also a 
random graph. Thus, the depth d k of the flow graph must also be a random variable. The objective 
of this section is to show the following proposition: 

Proposition 2: The depth d k of each flow graph Gk is 6(log N) with high probability. 

To prove Proposition we follow the following three steps. First, to characterize random 
variable d k , we need to characterize the random graph Gk- In this step, we show that there is an 
alternative way to construct the random graph which is stochastically equivalent to the construction 
described in Section [TV] In the second step, using the alternative construction, we will show that 
the number of peers within I hops from the source in Gk increases exponentially in I. In the last 
step, we relate this result to the depth of Gk- 

A. Distribution of Flow Graphs 

Consider two random multi-digraphs G' = (V, E') and G" = (V, E") that have the same peer 
set V and random edges. For every possible multi-digraph G with peer set V, we say that these 
two random graphs have the same distribution if P[G' = G] = P \G" = G] . In this subsection, 
we consider how flow graph Gk is distributed and how to construct a random graph that has the 
same distribution as flow graph Gk- 

Recall that flow graph Gk is the superposed graph of layer Xk and (V,£M,k), a subgraph 
of layer M. Thus, we first consider the distribution of each layer. As in the first example of 
Section [HI] construct a random Hamiltonian cycle by permuting the peers in V and let H denote 
this cycle. (Recall that all the graphs in this paper are directed graphs, and thus we omit repeatedly 
mentioning "directed.") Let C(V) be the set of all possible Hamiltonian cycles that we can make 
with peer set V. Then, it is easy to see that H is distributed as 

1 1 
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for every G G C(V). The following proposition shows each layer L m has the same distribution 
as H. 

Proposition 3: For m = 1, 2, • • • , M, each layer m (denoted by L m ) and random Hamiltonian 
cycle H have the same distribution, i.e., for every G G C(V), 

P[L m = G]=P[H = G]= l _ 

Proof: We prove by induction. Fix layer m (L m ). Initially (with two peers 1 and 2), layer 
m is 1 — > 2 — > 1, which is the only possible Hamiltonian cycle with two peers. Thus, each layer 
and H have the same distribution. 

Suppose that L m = (V,E m ) and H G C(V) have the same distribution for peer set V with 
|V| > 2. If a new peer i ^ V joins, this peer chooses one edge from L m uniformly at random 
and breaks into the edge. Let L' m be layer m after adding peer i. Similarly, choose one edge 
from H uniformly at random and add peer i into the edge. Let H' be the Hamiltonian cycle after 
adding peer i to H. Clearly, L' m has the same distribution as H'. It is easy to see that making a 
Hamiltonian cycle by permuting \V\ peers and then adding peer i into the cycle is equivalent to 
making a Hamiltonian cycle H" by permuting the peers in 1/ U {i}. Thus, H' and H" have the 
same distribution, and so do L' m and H". 

Suppose that L m = (V,E m ) and H £ C(V) have the same distribution for peer set V with 
\V\ > 2. We remove a peer i £ V from layer m and connect its incoming and outgoing edges. 
Let L' m denote the layer after removing the peer. Since L m and H have the same distribution, if 
we remove peer i from H, the resulting graph H' will also have the same distribution as L' m . It 
is easy to see that making a Hamiltonian cycle by permuting the peers in V and removing peer 
i is equivalent to making a Hamiltonian cycle H" by permuting the peers in V \ {i}. Thus, H' 
and H" have the same distribution, and so do L' m and H", 

By induction, at any given time, each layer L m with peer set V has the same distribution as a 
random Hamiltonian cycle obtained by permuting the peers in V. ■ 

Proposition [3] shows that conditioned on peer set V, each layer has the same distribution as H. 
Among layer A& and (V, £M,k) that form flow graph Gk, Xk can be replaced with H for analysis. 

We next consider how (V,£M,k) is distributed. Note that (V,£M,k) is a subgraph of layer M 
that consists of only the edges £ Em with fj,(i) = k. Since each peer i makes its coloring 
decision to be n(i) = k with probability q = 1/(K — 1), (V, £M,k) can t> e seen as the graph made 
from layer M by independently removing each edge with probability 1 — q. Since layer M has 
the same distribution as H, (V,£m.k) has the same distribution as the graph H' that we obtain 
from H by removing each edge with probability 1 — q. 

Note that conditioned on peer set V, layer and layer M are mutually independent because 
peer pairing in a layer has been independent from that in another layer, i.e., for any Q' , Q" C C(V), 

P [L Xk G Q', L M g G"\ = P [L Xk G Q'\ P [L M £ Q"] ■ 

Thus, the graph that has the same distribution as Gk can be constructed from two independent 
random Hamiltonian cycles as follows: 

Proposition 4: Construct two random Hamiltonian cycles Hi and H2 by permuting the peers in 
V independently for each. Remove each edge from H% with probability 1—q, where q = 1/(K—1), 
and call the resulting graph H' 2 . If we superpose Hi and H' 2 , the superposed graph H* has the 
same distribution as flow graph Gk for k = 1, 2, • • • , K — 1. 

Note that H* in Proposition |4] is identical to the graph in the second example of Section lull 
Thus, the maximum distance from a given node to all other nodes in that graph is stochastically 
equivalent to the depth of each flow graph. By proving the depth of H* is (log AT) with high 
probability, we show that the depth of flow graph is also 9 (log A) with high probability. 

There exists several ways to construct Hamiltonian cycles H\ and H 2 - The simplest way is 
to permute the peers and connect the permuted peers with edges. However, when we construct 
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Hi and H2 using this method, analyzing the depth of H* is not straightforward. We instead use 
another equivalent process which provides us with a tractable construction amenable to analysis. 

Directed edges are added to the graph one by one in the following manner: among the set of 
peers which do not have an outgoing edge, a peer is chosen arbitrarily to be the head of a new 
directed edge. Then, among the peers which do not already have an incoming edge, a node is 
chosen according to some pre-defined rule to be the tail of the new directed edge. Then, a directed 
edge is drawn from the head to the tail. This process is repeated till every node has exactly one 
incoming and one outgoing edge. It is not difficult to see that the resulting graph is a Hamiltonian 
cycle. Note that successive edges produced by this process need not have a consecutive edges on 
the Hamiltonian cycle. In other words, we could draw (vi,ci) first and draw (i>2>C2) for some 
peer V2 7^ c\. We formally state this cycle construction process next. First, we state a condition 
for a set of nodes and edges to be the subgraph of a Hamiltonian cycle. Then, we present the 
process for constructing a random Hamiltonian cycle. 

Condition 1: For (V,E) to be a subgraph of a Hamiltonian cycle, 

(a) Each peer has at most one incoming edge and at most one outgoing edge. 

(b) There is no cycle if \E\ < \V\. 

It is easy to see that if we sequentially draw N edges satisfying these conditions at each step, 
the resulting graph becomes a Hamiltonian cycle. Now, we describe the process for generating a 
random Hamiltonian cycle: 
Cycle Construction Process: 

1) Initially, take E(0) = 0, vi = s (source), and t = 1. 

2) At iteration t > 0, pick a peer v t from V \ {i>i,i>2, • • • ,vt-i} according to some specific 
rule, i.e., we assume that there is a function which takes an element of the set {vi, . . . , v t -i} 
as input and produces an output which is an element of the set V \ {vi, . . . , Vt-i}. 

3) Let C(v t ,E(t - 1)) be the set of peers j such that E = E(t - 1) U {(v t ,j)} satisfies 
Condition [T] 

4) Pick a peer ct from C(vt,E(t — 1)) uniformly at random and add (v t , c t ) to the graph, i.e., 

E(t)=E(t-l)U{(v t ,Ct)}. (3) 

5) If t < N, increase t by one and go to Step 2. 

Since we have drawn an edge at each iteration satisfying Condi tionQ] the resulting graph (V, E(N)) 
must also be a Hamiltonian cycle. Further, the following proposition shows that the resulting 
Hamiltonian cycle has the same distribution as Hi. 

Proposition 5: The resulting graph (V, E(N)) from the cycle construction process has the same 
distribution as random Hamiltonian cycle H\. 

Proof: Fix a Hamiltonian cycle G G C(V). To show that (V,E(N)) and Hi have the same 
distribution, we need to show that P[(V,E(N)) = G] = l/(N - 1)! by ©. We prove this by 
induction. First, assuming the fixed Hamiltonian cycle G to be G = (V, E), we show that the 
graph constructed right after iteration t is a subgraph of the Hamiltonian cycle with probability 

* 1 

P [E(t) C E\ = TT — -, forl<t<7V-l. (4) 

Once the above is shown, we have P [E(N — 1) C E) = l/(N — 1)1. In other words, if we draw 
N— 1 edges using the cycle construction process, the resulting graph (V 1 E(N—1)) is a subgraph of 
the Hamiltonian cycle G with probability 1/(N — 1) !. There is only one way to draw the iV-th edge 
at the last iteration, which makes (V, E(N)) to be G. Therefore, P [(V, E(N)) = G] = 1/(JV-1)!, 
and thus the resulting graph has the same distribution as Hi. 

We now focus on showing (O. At t = 1, say (vi, c) G E. The cycle construction process draws 
(vi,ci), choosing c\ from C(vi,E(0)) uniformly at random. Since E(Q) = 0, it is easy to see 
that there are N — 1 peers in C(vi, E(Q)). Thus, the probability that peer c is chosen as ci is 
l/(iV - 1). Therefore, © is true for t = 1. 
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We now assume that <j4j is true for t < I. We show that (0]i is also true for t = I. At iteration 
I, the cycle construction process draws (vi,ci), choosing c; from C(vi,E(l — 1)) uniformly at 
random. For {(vi, ci)}\JE(l — 1) to satisfy ConditionQ] q must not be in V 1 = {ci, C2, • • • , Q_i}. 
Hence, only the peers in V \ V can be c;. If w; € V', there must exist a peer i ^ V' such 
that there is a path from i to i>;. If we add (vi,i) to £7(i — 1), this path will become a cycle, 
which contradicts to Condition [T] Hence, such peer i must not be in C(vi,E(l — 1)). Thus, 
\C(v h E(l - 1))| = N - (I - 1) - 1 = N - I if vi e V. If £ V, peer vj itself must not be 
in CiC(vi, E(l — 1)). (Otherwise, it will make a loop, which is a length-0 cycle.) In this case, 
\C(vi, E(l — = N — (I — 1) — 1 = N — I, Overall, peer q is chosen uniformly at random 
among N — I peers. Say (vi, c) 6 E. By the induction hypothesis, 



From this result, the cycle construction process can be seen as another method to construct H±. 

We next consider a process that iteratively constructs H 2 . Recall that H 2 is the graph made 
from H 2 by removing each edge with probability 1 — q. Thus, at each iteration of the cycle 
construction process, we add (vt,Ct) to E(t) using (0 with probability q = 1/(K — 1) and do 
not add the edge to E(t) (i.e., E(t) = E(t — 1)) otherwise. Then, the resulting graph from this 
modified cycle construction process satisfies the following property: 

Lemma 3: The resulting graph (V,E(N)) from the modified cycle construction process has 
the same distribution as H' 2 . 

We finally consider a process that iteratively constructs the superposed graph H* of H\ and 
H' 2 . One simple way is to construct Hi and H' 2 separately from the two processes and combine 
them. However, analyzing this construction appears to be difficult. Instead, we now present an 
unified process that constructs Hi and H 2 simultaneously as follows: 
Flow Graph Construction Process: 

1) Initially, Z(0) = s, E(0) = E'(0) = 0, v x = s, and t = 1. 

2) At iteration t, among the peers in V \ {vi, v 2 , ■ ■ ■ , Vt-i}, pick the peer that was first added 
to Z(t — 1) and call it peer vt- 

3) Choose ct and c' t from C(vt,E(t — 1)) and C(v t ,E'(t — 1)), respectively, uniformly at 



4) Update E(t) = E(t - 1) U {(v t ,c t )}. 

5) Define a variable fi t as follows: with probability q, let \it = 1 and with probability 1 — q, 
let fi t = 0. 

6) If fi t = 1, update E'{t) = E'{t - 1) U {(v t ,c' t )} and Z(t) = Z(t - 1) U {ct, d t }. 

7) If fit = 0, update E'{t) = E'(t - 1) and Z(t) = Z(t - 1) U {c t }. 

8) If t < N, increase t by one and go to Step 2. 

If we follow the above steps to draw the edges in E(N) in the flow graph construction process, 
we can see that these steps are the same as those in the cycle construction process except for the 
specific choice of vt at each iteration. While v t is chosen arbitrarily from V \ {vi,v 2 , ■ ■ ■ , v t -i} 
in the cycle construction process, vt is chosen from the same set according to a specific rule (Step 
2) in the flow graph construction process. Thus, constructing graph (V,E(N)) in the flow graph 
construction process can be seen as a special case of the cycle construction process. Thus, graph 
(V,E(N)) from the flow graph construction process must have the same distribution as Hi by 
Proposition 



P[E(1)CE] 

= P [(«,, Cl ) e E I E{1 - 1) C E] P [E{1 -1)CE] 
= P[ci = c\E(l-l)cE]P [E{1 -l)cE] 




Therefore, (|4]i is true for t = I. By induction, (HJi is true for 1 < t < N. 



random. 
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Fig. 5. If |Z(t)| > (1 + ip)t for k <t < N/2 and some ip > in the flow graph construction process, the number of 
the peers within /-hops from the peers v\,V2, ■ ■ ■ is (1 + ip) 1 k, which increases exponentially in I. Thus, N/2 peers 
can be covered within O(logTV) steps. 

Similarly, if we follow the steps to draw the edges in E'(N) in the flow graph construction 
process, we can see that these steps are the same as those in the modified cycle construction 
process except for the specific choice of v t at each iteration. Thus, constructing graph (V, E'(N)) 
in the flow graph construction process is a special case of the modified cycle construction process. 
Hence, (V,E'(N)) from the flow graph construction process has the same distribution as H' 2 by 
Lemma [3] Overall, (V,E(N) U E'(N)) that we have drawn in the flow graph process will have 
the same distribution as H* by Proposition @] Thus, the flow graph construction process can be 
seen as an another way to construct flow graph Gk- Note that we do not propose this flow graph 
construction process to construct the network topology in practice. We use this flow graph process 
for analysis and use our peer-pairing algorithm in practice, which results in random graphs with 
the same distribution. In the rest of this section, we consider the depth of flow graph Gk from 
the depth of H* in the flow graph construction process. 

B. Edge Expansion of Flow Graph Gk 

As a preliminary step to find the depth, we consider how H* expands and how this is related 
to the depth of the graph. Suppose that there exists ip > that satisfies 

\Z{t)\ > (1 + ij))t, for k < t < N/2. (5) 

In other words, if we group peers vi,V2, ■■■ ,v t in the flow graph construction process as a set, 
there are at least ipt edges going out from the set as shown in Fig. [5] Recall that Z(t) is the set of 
the first t peers and their children in the flow graph. Thus, if Sq denotes the set of the first k peers, 
there are at least ipk outgoing edges. If Si = Z(k), Z(\Si\) is the set of the peers in Si and their 
children. (Note that Z(t) = {vi, 1)2, • ■ ■ , v\z(t)\} because the order vi, V2, • • ■ , wjv is determined 
by the order that the peers are added to Z(t) in the flow graph construction process.) By (0, Si 
has at least V'l'S'il outgoing edges, which is at least ip(l + ijj)k. If 5*2 denotes the set of the peers 
in Si and their children, it is satisfied that IS2I > (1 + ip) 2 k. Following the same procedure, we 
can obtain that > (1 + ijj) l k for I > 0, which means the number of the peers that are within 
Z-hops from the first k peers increases exponentially, as in Fig. [5] Therefore, to cover N/2 peers, 
it needs only 0(logiV) steps, i.e., jv) > N/2. (We will see that covering the other N/2 

peers also takes O(logTV) steps.) Thus, if the flow graph construction process satisfies (0, the 
graph must have O(logiV) depth. 
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To show the expansion of flow graph, we focus on how Z(t) increases. For simplicity, let 
z(t) = \Z(t)\. At each iteration t, if peer q is not in Z(t — 1), adding this peer to Z{t) will 
increase z(t) by one from z(t— 1). In addition, if fi t = 1 and peer c' t is not in Z(t) = Z(t— l)U{c{}, 
adding peer c' t will also increase z(t) by one. Hence, the increment of z(t) at iteration t is given 
by 

Z{t) = Z(t - 1) + l{c t £Z(t-l)} + fit ■ l{c^Z(t-l)u{c f }} (6) 

At each iteration, the increment of z(t) is either 0, 1, or 2. If it is 2, z(t)/t increases because t 
increase by one. In contrast, if the increment is or 1, z(t)/t decreases. Initially, z(t) increases by 
two with high probability because Z(t) contains a few peers compared with V. Therefore, z(t)/t 
increases with high probability. As t increases, i.e., as Z(t) contains more peers, the probability 
that the increment is or 1 increases, and thus z(t)/t will decrease more frequently. The following 
proposition shows how the mean of z(t) evolves. 
Proposition 6: For < I < t, 

E[N - z(t)\E(l),E'(l)] 

« W-,<„,. ,7, 



N- I - 1 V N- \E'{1)\ - 1 

The proof is provided in Appendix lAl 

From Proposition [6] E[z(t)] can be obtained by taking I = as follows: 



E[N-z(t)] =(N-t-l)\l 



tq 



N - 1 



.«U 1 + I +9(1 __1_, 

For t < N/2, we can see that the minimum of E[z(t))/t is attained at t = N/2, and the minimum 
is greater than 1 + q/2. Thus, we have 

E\zit)] q N 

In other words, for the set of peers Vi,i>2, • ■ • ,vt, the expected number of outgoing edges from 
the set is at least ipt. 

To show the expansion of our graph as in Fig. [5] we need z(t)/t > l+q/2 with high probability. 
Thus, we consider how much z(t)/t is concentrated around its mean. Using a Doob Martingale 
and the Azuma-Hoefdling bound, we have the following result: 

Proposition 7: For ip S (0, |) and t < 

P[z(t)>(l+4>)t]>l-exp(-<rt), (8) 

where q = 1/(K - 1) and a = 

The proof is provided in Appendix [Ijj 

From this result, if we consider the set of first t peers in the flow graph, the set has at least ipt 
outgoing edges with high probability. However, we cannot directly apply this result to the example 
in Fig. [5] and conclude that the distance to the closest N/2 peers from the source is Q(\ogN) 
because it is unclear that Proposition [7] holds for multiple sets So, Si, S2, • • ■ simultaneously. We 
next consider how to relate Proposition [7] to the actual depth of flow graph Gk- 



C. Depth of Flow Graph Gk 

In this subsection, we finally show that the depth of flow graph Gk is (log TV) using all 
aforementioned results. Recall that dk{v) is the distance from the source to peer v in Gk- Let 
d(v) be the distance from the source to peer v in the random graph H* constructed by the flow 
graph construction process. Since Gk and H* have the same distribution, the distributions of 
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the random variables dk{v) and d(v) must agree. Thus, we can focus on showing that d(v) is 
0(logiV) with high probability. 

From the flow graph construction process, it is easy to see that d(vf) < d(vt") if t' < t" . Since 
Z(t) is the set of the first t peers and their children, it is clear that max v£ z(t) d(v) — d(vt) < 1. 
Note that v z m is the peer with the maximum distance in Z(t) because Z(t) = {v\, V2, • • ■ , v z {t)}- 
Thus, we have 

d(v z{t) ) - d(vt) < 1, for < t < N. (9) 

Using this property and Proposition [7] we can derive the distance from the source to peer Vjv/2> 
the median distance from the source to all peers in H*, as follows: 

Proposition 8: Let d m be the median distance from the source to all peers in H*. For ip 6 

(0,1), 

AH 



p 



d m < log N + log 1+v , — 



> 1 - 



lQ gi+^ f 



(10) 



where q = 1/(K - 1) and a = ( grff ■ 

Proof: For simplicity, let 4> = 1 + -0. Define event A t — {z(</>* log N) > 4> t+1 \ogN}. If 
event A t is true, we can derive the following from ©: 



d(v$t+i log jv) — d^v^t i g at) 
If A t is true for t = 0, 1, 2, • • ■ ,to — 1, we obtain 

d(w01ogAr) - d(«logJv) < 1 
d(^ 2 logiv) -d(«01ogjv) < 1 



< 1. 



By taking t = log,/, y — log,/, log A 7 ", we have 



d(«<^*o logw) - d((f> to ~ 1 vi ogN ) < 1 

/>'o log At) — ^(wiogjv) < to 

1>*o log w) < *o + d(v\ ogN ) <t Q + log A 7 



AT 



d(vN_) <\ogN + log y - log log A 7 ". 
Using a union bound, the probability that we are interested in can be expressed as 



P 



N 

d{VN ) < log N + log — - log log AT 



to-l 



> p [r^A t ] =i-p [ut^Af] >i-J2 p [ A t\ 



Since P [Af] < exp(— cr(</>* log AQ) by Proposition [7] the above inequality can be expressed as 



P 



N 

d(vN ) < log AT + log — - log(2 log N) 

to-l 



> 1 - cx p ( - o-iogA/^*) = 1 - X] iv_CT0t 



t=o 



> 1 



lo 



JV 
60 2 



Since Ui,^,- - • are sorted by their distance from the source, d(vN_) is the median distance from 
the source to all peers in H* that is constructed by the flow graph construction process. Thus, we 
have the result in the proposition. ■ 
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From Proposition [8] the median distance of O(logTV) implies that there must be at least N/2 
peers within 0(\ogN) hops from the source. However, only with the flow graph construction 
process, we cannot find the distance to the remaining peers because the key result in Proposition [7] 
is not valid for i > -y. To analyze the remaining distance, we consider H* with reversed edges. 
Specifically, if we make a graph H* by reversing the direction of each edge in H*, the distance 
from the source to peer v in H* is the same as the distance from peer v to the source in H*. 
From Proposition [8] the median distance from all peers to the source in H* is 0(log N) with high 
probability. This means that there are at least N/2 with high probability such that the distance 
from these peers to the source in H* is O(logiV). This property must be satisfied if we replace 
the source with any arbitrary peer, say v* 6 V. By showing that H* and H* have the same 
distribution, we have the following result. 

Lemma 4: Fix an arbitrary peer v* S V. Let d m be the median distance from all the peers to 
peer v* in H*. For ip <E (0, §), 



P 



d m < log N + log! 



(9/2-V-) 2 



N 



+1> 



> 1 



lo: 



>l+i/> 2 



N- 



where q = 1/(K - 1) and a = 2 (3- q ) i 

Proof: Since we have already shown that this property is satisfied in H*, we only need to 
show that H* and H* have the same distribution. Let Hi an H 2 be the graphs that we obtain from 
H\ and H 2 , respectively, in Proposition |4] by reversing the direction of each edge. Even though 
we have reversed the edges of both graphs, the resulting graphs have the same distribution, i.e., 
for every G £ C{V), 



P 



Hi = G 



P 



H 2 =G 



1 



(N-l)\ 



Let H 2 denote the graph we obtained from H 2 by independently removing each edge with 
probability 1 — q as we did for H 2 . Since H 2 and H 2 have the same distribution, so do H 2 
and H 2 . Therefore, if we superpose Hi and H 2 , we have the superposed graph H*, and this 
graph has the same distribution as H*, which is the superposed graph of Hi and H 2 . ■ 
We now complete the proof of Proposition |2] which is the main result of this section. Let v* 
be the farthest node from the source in H*. In H*, there are N/2 peers that the source can reach 
within O(logiV) hops with high probability by Proposition [8] and there are N/2 peers that can 
reach peer v* within 0(log N) hops with high probability by Lemma [4] Since there are N peers 
in the network, there must be a path of length O(logiV) from the source to peer v* with high 
probability. Using the union bound, the depth d* k of flow graph Gk, which corresponds to the 
distance from the source to the farthest peer v* in H* , is distributed as follows: 



P [d* k <26]>P 



d m < 29 



> P 



dm. < 0, d m < 



> 1 - P [d, n >0}-P 



> 



> 1 



2 log! 



N 

+i> 2 



N a 



(11) 



where 6 = log TV + log 1+ y, Similarly, the maximum depth d* of all [K — 1) flow graphs is 
distributed as 



P [d* <26]>l- 



2(if-l)log 1+ , | 
N a 



which is the result of Proposition [2] Thus, we conclude that every chunk arriving at the source 
at rate (1 — K _ t ) is disseminated to all peers within 0(log N) timeslots with high probability. 
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VII. Diameters of Flow Graphs 



Previously, we have shown that the maximum distance from the source to other peers in each 
flow graph is (log TV) with high probability. Although this result is enough to show 0(logA) 
streaming delay, we can prove the following stronger result which was mentioned in the second 
example of Section [TlTl 

Proposition 9: The diameter of each flow graph is ©(log A) with high probability. 

Proof: Let H* denote a flow graph, and let di.j be the distance from peer i to peer j. Let 
H* be a graph with the same set of edges as H* but with the direction of each edge reversed, 
and let dij be the distance from i to j in H*. Since the shortest path from i to j in H* is the 
shortest path from j to i in H* , it follows that dij ~ dj } i for all pairs From (fTTT i. 



P 



max dj , > 29 



P 



maxcL , > 29 



where 9 = log N + log 1+ ^, y. As we proved in Lemma [4] the random graphs H* and H* have 
the same distribution. Thus, the corresponding di .j and di.j also have the same distribution, i.e., 



P 



max d-j s > 29 

jev J ' 



P 



max dj s > 29 



= o(l). 



Using the union bound, we have 



P 



max dj s < 29 and max d s , 



< 29 



=1-P 



>1-P 
=l-2-o(l) 



max rf., s > 29 or max d s j > 29 



max dj s > 29 
l-o(l) 



-P 



max d s ,■ > 29 



This result implies that for any two peers i and j, both the distance from i to the source and the 
distance from the source to j are 0(log N) with high probability. Thus, the distance from i to j 
must be 0(log N) with high probability. ■ 
Since Proposition requires the distance between any pair of peers to be 0(log JV), this result is 
much stronger than Proposition|2] which requires only the distance from the source to be 0(log N). 
This stronger result allows us to extend our P2P network with a single source to that with multiple 
sources. Suppose that there are K — 1 sources, each of which wants to broadcast its own video 
content at rate 1/K to all peers. In other words, assume that the fc-th source (k < K) generates 
a color-fc chunk once every K timeslots. If peers use the same chunk-dissemination algorithm 
described in Section [IV] color-fc chunks will be delivered through flow graph Gf., which has been 
proved to have a diameter of O(logiV) with high probability. Thus, each color-fc chunk generated 
by the fc-th source can be delivered to all peers within O(logiV) timeslots with high probability. 



VIII. Conclusions 

Instead of conventional approaches using multiple overlay trees, we have proposed a simple P2P 
streaming algorithm that consists of a simple pairing algorithm proposed earlier for constructing 
distributed hash tables |fl~), but used here for streaming data. Our proposed chunk dissemination 
algorithm can deliver all chunks to all peers with (log AT) delay for a (1 — 1/AT) of the optimal 
streaming capacity for any constant K > 2. 

There are several issues that need to be addressed to implement our algorithm in practice. 
The first issue is one of modifying our chunk dissemination algorithm to accommodate peer 
churn. Even though our chunk dissemination algorithm shows that the network topology at any 
given moment can achieve both a near optimal throughput and ©(log A) delay, peer arrivals and 
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departures disrupt the topology continuously and hence, one needs practical solutions to account 
for this churn in the chunk dissemination algorithm as well as the delay analysis. The second issue 
is one of dealing with packet losses. Even in the wired Internet, packet losses are not uncommon, 
and therefore, a practical protocol must have provisions to recover from such losses. Finally, we 
have to deal with asynchronous transmissions, i.e., chunk transmissions will not occur in a time- 
slotted manner in the Internet for many reasons. The analysis gets much more involved in this 
case. Dealing with these practical issues is an important avenue for future work. 

References 

[1] C. Law and K.-Y. Siu, "Distributed construction of random expander networks," in Proceedings of IEEE INFOCOM, 

vol. 3, April 2003, pp. 2133 - 2143 vol.3. 
[2] Y.-J. Chu and T.-H. Liu, "On the shortest arborescence of a directed graph," Science Sinica, vol. 14, pp. 1396-1400, 

1965. 

[3] J. Edmonds, "Edge-disjoint branchings," Combinatorial Algorithms, pp. pp. 91-96, 1972. 

[4] R. E. Tarjan, "Finding optimum branchings," Networks, vol. 7, no. 1, pp. 25-35, 1977. [Online]. Available: 

http://dx.doi.org/10.1002/net.3230070103 
[5] J. Li, P. A. Chou, and C. Zhang, "Mutualcast: an efficient mechanism for one-to-many content distribution," in 

Proceedings of ACM SIGCOMM, Beijing, China, April 2005. 
[6] J. Mundinger, R. Weber, and G. Weiss, "Optimal scheduling of peer-to-peer file dissemination," J. of Scheduling, 

vol. 11, pp. 105-120, April 2008. [Online]. Available: [http://m.a cm.org/citati onTfm?id=134963034 9650 
[7] R. Kumar, Y. Liu, and K. Ross, "Stochastic Fluid Theory for P2P Streaming Systems," in Proceedings of IEEE 

INFOCOM, Anchorage, AL, April 2007. 
[8] S. Liu, R. Zhang-Shen, W. Jiang, J. Rexford, and M. Chiang, "Performance bounds for peer-assisted live streaming," 

in Proceedings of ACM SIGMETRICS, Annapolis, MD, 2008, pp. 313-324. 
[9] S. Liu, M. Chen, S. Sengupta, M. Chiang, J. Li, and P. A. Chou, "P2p streaming capacity under node degree bound," 

in Proceedings of IEEE ICDCS, 2010, pp. 587-598. 
[10] C. Zhao, X. Lin, and C. Wu, "The streaming capacity of sparsely-connected p2p systems with distributed control." 

in Proceedings of IEEE INFOCOM, 2011, pp. 1449-1457. 
[11] X. Zhang, J. Liu, B. Li, and Y.-S. Yum, "Coolstreaming/donet: a data-driven overlay network for peer-to-peer live 

media streaming," in Proceedings of IEEE INFOCOM, vol. 3, 2005, pp. 2102 - 2111. 
[12] A. M. Frieze and G. R. Grimmett, "The shortest-path problem for graphs with random arc-lengths," Discrete Applied 

Mathematics, vol. 10, no. 1, pp. 57 - 77, 1985. 
[13] S. Sanghavi, B. Hajek, and L. Massoulie, "Gossiping with multiple messages," in In IEEE Transactions on Information 

Theory, 2007, pp. 4640-4654. 

[14] T. Bonald, L. Massoulie, F. Mathieu, D. Perino, and A. Twigg, "Epidemic Live Streaming: Optimal Performance 

Trade-Offs," in Proceedings of ACM SIGMETRICS, Annapolis, MD, June 2008. 
[15] D. Shah, Gossip Algorithms. Foundations and Trends in Networking, 2009, vol. 3, no. 1, pp. 1-125. 
[16] J. H. Kim and N. C. Wormald, "Random matchings which induce hamilton cycles and hamiltonian decompositions 

of random regular graphs," Journal of Combinatorial Theory, Series B, vol. 81, no. 1, pp. 20 - 44, 2001. [Online]. 

Available: http://www.sciencedirect.com/science/article/pii/S0095895600919919 
[17] B. Bollobas and W. Fernandez de la Vega, "The diameter of random regular graphs," Combinatorica, vol. 2, pp. 

125-134, 1982, 10.1007/BF02579310. [Online]. Available: [hTtp://dx.doi.org7i^0 07/BF02579310l 
[18] J. Kim and R. Srikant, "P2P Streaming Over Dynamic Random M -Regular Graphs," Technical Report, 2011, 

http:// www. ifp. illinois. edu/~ joohwan/Kiml ltech2.pdf 
[19] L. Massoulie, A. Twigg, C. Gkantsidis, and P. Rodriguez, "Randomized decentralized broadcasting algorithms," in 

Proceedings of IEEE INFOCOM, 2007, pp. 1073-1081. 



Appendix 

A. Proof of Proposition \6\ 

We prove this result by induction. We first show that © is true for I = t. Since the edge sets 
E{t) and E'(t) fully characterize the graph that have been drawn by iteration t, Z(t) and z{t) 
are deterministic. Thus, we can remove the expectation from N — z(t), which corresponds to (j7]i 
for I = t. 

We assume that (O is true for I = k + 1 < t, i.e., 

E[N -z(t)\E(l),E'(l)] 

N-t-l f (t-(k + l))q \ 

N-(k + l)-l\ N - \E'(k + 1)| - 1/ X 

{N-z{k + l)). (12) 
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We then show that (Q is also true for I = k. For simplicity, multiply both sides of dT2b by 
If we replace z(k + 1) using ©, we have 

N — k — 2 

N _ t _ l E i N - *(*) I E ( k + l )> E '( k + !)] 
=f(k) N - z(k) - l {ct ^z(fe)} - (J-kl{c' t <tz(k)u{c t }} 

where 

/(*) = ! (' -(* + !))* 



(13) 



If we take the conditional expectation E[ ■ | E(k), E 1 (k)] on both sides, 

N — k — 9 

———E[N - z{k)\E{k) 1 E'(k)] 

=E{f(k)(N-z(k))\E(k),E'(k)} 
-E[f(k)l {Ck+imk)} \E(k),E'(k)} 
-E[f(k)fi k+l l Wh+imk)u{ck+l}} | E(k),E'(k)\. 

In the first term, N — z(k) is deterministic conditioned on E(k) and E'(k). In the second term, 
whether or not the peer c k+ i belongs to Z(k) is independent of the coloring decision fik+i in f(k). 
Hence, f(k) and l{ Ck+1 fz(k)} are independent. In the last term, f(k)fi k+1 and l{ c ' fc+1 £z(fc)u{ Cfc+1 }} 
are independent because c' k+1 is chosen independently from (i k +i. Hence, ( fT3] l can be rewritten 

as 

N — k — 9 

———E[N - z(k)\E{k) 1 E'{k)] 

=(N - z(k))E[f(k) | E(k), E'(k)] 

-E[f(k) | E(k),E'{k)]-E[l {ck+imk)} | E{k),E'{k)] 

-E[f(k)fi k+1 | E(k),E'{k)]x 

E[l Wk+imk )uW k+l}} \E(k),E\k)}. (14) 

We first find E[f(k) \ E(k),E'(k)]. Since fj, k+ \ = 1 with probability q = and fj, k+1 = 

otherwise, independently of E(k) and E'(k), we have 

E[f(t)\E(k),E'(k)] 
(t-(k + l))q 



=3U 



+ (1 - 9) 1 



iV- |£'0)| -2 



iV - |-B'(fc)| - 1 
g(i-fe-l) 



iV - \E'(k)\ - 1 

g 2 (< - k - 1) 
" (iV-|£;'(fc)|-l)(JV-|£'(fe)|-2)- 

Similarly, we have 

E[f(k)fi k+1 \E{k),E'{k)\ 

_L (t-(k + l))g \ _ g g 2 (t-fc-l) 



iV-|S'(fc)|-2y * TV - |£"(fc)| - 2 ' 

We next derive an expression for E[l{ Ck gz(k)} I E(k), E'(k)]. Note that c^+i is chosen 
uniformly at random from C{v k +\, E(k)), which has N — (k + 1) peers as shown in the proof 



21 



of Proposition [5] Thus, the conditional mean of the indicator function is equal to 

E[l {ck+imk)} \E(k),E'(k)} = ^Zl% 

Similarly, we can derive an expression for ^[l{ c ' fc+1 ^z(fe)u{c fc+1 }} I E(k), E' (k)]. Since c' k+1 is 
chosen uniformly at random from C(vk+i, E'(k)), which has N — \E'(k)\ — 1 peers. Thus, the 
probability of c' k+1 £ Z(k) U {ck+i} conditioned on E(k), E'(k), and c k+ i is 

P [c' k+1 i Z(k) U {c k+1 }\ E(k),E'(k),c k+1 ] 

n - \z(k) u {c k+1 }\ _ n- z(k) - i {Ck+1 mm 

N — \E'(k)\ — 1 N-\E'(k)\-l 
Removing the conditioning with respect to c k+ i, we have 

E{l Wk+imk)uWk+1 }} \ E(k),E'(k)} 
_ N - z(k) - E[l Wk+imk)} | E(k), E'(k)} 
N - \E'(k)\ - 1 

_ (N-z(k))(§E^) 
N — \E'(k) \ — 1 ' 
Applying the above to ( TT4l i. we can simplify ( fT4b to 

E[N-z(t) | E(k),E'(k)] 



N -k- 1 V l^'(fc)l - 1 

Thus, © is true for Z = fc. By induction, © is true for 1 < I < t. 

B. Proof of Proposition [7| 

We first define the Doob martingale {B k } < k < t as B k = E[z(t) \ E(k),E'{k)]. From ©, 
we find the upper bound ip k of \B k — B k -i\ for each k and then find the probability that the 
martingale concentrates around its mean using the Azuma-Hoeffdling inequality. 

The upper bound <j> k can be found from the following lemma. 

Lemma 5: \B k - B fe _i| < 3 - q, for t < N/2, 
The proof is provided in Appendix [C] Using the Azuma-Hoeffding bound with this Lipschitz 
bound, we have 

2 

P[B t -B >a]<cx P (- ° - ). (15) 

From Proposition |6l B t = N — z(t) and Bq = (TV - 1 - 1)(1 - r^ziy). Taking a = 1 + i(g(l - 
(fT~5T > can be expressed as follows: for t < N/2, 

2 

P [>(*) < (1 + ip)t] < cxp f j— ) 

<eXP (-|(3^F) (16) 
2(3 - qf 

In (Unj), we have used a < (q/2 — ip)t for t < N/2. Thus, we have proven Proposition [7] 



N-l' 



ex P ( ~ ~iuo 32") = exp(-at). 
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C. Proof of Lemma \5\ 

From Proposition [6] we have 

Bk — Bk-i 
N-t-1 




(t - k)q 



{t-k + l)q 



N 



(N-z(k)) 



(N-z(k-l)) 



(17) 



By differentiating Bk — -Bfc-i with respect to each Hj for j = 1, 2, • • • , k, we can see that 
Bk — Bk-i is maximized with fik = and fj,j = 1 for j < k and minimized with \ik = 1 and 
Hj = for j < k. Further, since N — z{k) < N — z(k — 1), Bk — -Bfc-i is upper bounded as 



<- 



B k 
N ■ 



B k -i 
t - 1 



N-k-1 
N -t 



(t-k)q 
N -k 



(N - z(k - 1)) 



N - k 



(t-k + l)q 



N-k 



< 



(N -t-l){N - z{k-l)) 



N 



(N - z(k 
-(t-k)q 



1)) 



N-k 
N — k — (t — k + l)q 



N 



N - 

The term in the bracket can be simplified as 

z(k — 1) < k, we have 



(JV-fc)(l- g )-(t-fc+l)q 
(N-t)(k-k-l) 



< 



1-9 

N-k-1 



Bk-B^<§-^(l-q). 



Applying this with 



(18) 



We next consider the lower bound. Since Bk — B^-\ is maximized with ^ = 1 and [ij = 
for j < k, we have 



>- 



Bk — -Bfc-i 
N-t-1 
N-k-1 
N-t-1 
N-k 



1 



(t-k)q 
N -2 
(t - k 



(N - z(k - 1) - x(k)) 
-IV 



N -1 



> 



N-t—lN — 2 — (t — k)q 
N-k-1 N-2 

N — t — 1 N — 1 — (t — k + l)q 



N-k 
N -t 



1 



N 
■x 



1 



(JV — z(k - 1)) 
(N — z(k — 1) — 2) 

(JV - z(k - 1)) 



(JV-fc-l)(JV-2) 
(JV - 2 - (t - k)q)(N - z(k - 1) - 2) 

-(JV-l-(t-/c + l)g)(JV - z(fc - 1)) 

The term in the bracket can be simplified as —2(N — 2 — (t — k)q) — (1 — q)(N — z(k — 1)). 
Since both (JV — 2 — (t — k)q) and (JV — z(Jc — 1)) are lower bounded by JV — 2, the simplified 
term is lower bounded by (—3 + q)(N — 2)). Hence, we have 



Bi. — B 



k-l 



> 



N-t-1 
N-k-1 



(-3 + «). 



(19) 
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From < fT8l > and $1% , the upper bound ip^ of \Bk — -Bfc-i| is given by 

** = £^(3-*)<(3- ff ). 



